The information technology (IT) industry is in the midst of a fundamental change toward centralization of application delivery via concentrated and dense private and public data center cloud computing sites.
A data center is a facility used to house computer systems and associated components (i.e., storage systems). Data centers typically provide high reliability and security. Data centers are often shared resources used by multiple clients of the data center operator. Large data centers are industrial scale operations using as much electricity as a small town.
Data centers are experiencing a new type of traffic pattern that is fundamentally changing design plus latency and performance attributes. Not only are traditional north-south or client-server flows growing but east-west or server-server flows now dominate most private and public datacenter cloud facilities.
Equipment manufacturers are now designing equipment that is specially designed to be used in data centers. This equipment is capable of forwarding data traffic with ultra-low latency and maximum throughput. 10 gigabit Ethernet switches produce 400 to 700 nanoseconds of latency. It is anticipated that 100 gigabit Ethernet switching will reduce latency to as low as 100 nanoseconds. In special applications, such as high frequency trading and other financial applications, differences in latency can result in millions of dollars in revenue lost or gained, placing enormous pressure on networking devices to be as fast as engineers can design.
Therefore, data center managers are facing many critical challenges in designing a data center solution that meets their business needs. Their dilemmas include determining how to test/validate the performance and resiliency of their network and evaluating the performance of devices from different vendors. Data center managers often lack sufficient knowledge and expertise to run various test methodologies which can validate the performance of a device under test (DUT). Ultimately, data center managers want to build a data center network using devices that excel in performance and maximize return on their investment.
Tests available from test system manufacturers can individually test performance metrics, such as jitter, throughput, and latency and provide separate results from each test. However, individually, these performance metrics do not characterize the overall performance of a device under test, and test results may vary greatly depending on test setup conditions. Even assuming that test conditions used to evaluate devices from different vendors are the same, if a device from vendor A has a higher throughput test result (high throughput is desirable) than a device from vendor B but also has a higher jitter or latency test result (high jitter and high latency are undesirable), the data center manager will still not know which device has the better performance because the data center manager may not know how to characterize the relative importance of the latency, jitter, and throughput test results.
In addition, even for a single parameter, such as latency, there are multiple different standardized latency tests. If vendor A's device performs better than vendor B's device in one standardized latency test but poorer in another standardized latency test, the data center manager is again faced with the problem with how to characterize the relative importance of the two different standardized tests.
Accordingly, in light of these difficulties, there exists a long felt need for methods, systems, and computer readable media for performing benchmark reference testing of data center and network equipment.